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ABSTRACT 

Ares l-X is a pathfinder vehicle concept under development by NASA to demonstrate a new class of 
launch vehicles. Although this vehicle is essentially a shell of what the Ares I vehicle will be, efforts are 
underway to model and calibrate the analytical models before its maiden flight. Work reported in this 
document will summarize the model calibration approach used including uncertainty quantification of 
vehicle responses and the use of non-conventional boundary conditions during component testing. 

Since finite element modeling is the primary modeling tool, the calibration process uses these models, 
often developed by different groups, to assess model deficiencies and to update parameters to reconcile 
test with predictions. Data for two major component tests and the flight vehicle are presented along with 
the calibration results. For calibration, sensitivity analysis is conducted using Analysis of Variance 
(ANOVA). To reduce the computational burden associated with ANOVA calculations, response surface 
models are used in lieu of computationally intensive finite element solutions. From the sensitivity studies, 
parameter importance is assessed as a function of frequency. In addition, the work presents an approach 
to evaluate the probability that a parameter set exists to reconcile test with analysis. Comparisons of pre- 
test predictions of frequency response uncertainty bounds with measured data, results from the variance- 
based sensitivity analysis, and results from component test models with calibrated boundary stiffness 
models are all presented. 


1 INTRODUCTION 


The process of model calibration involves reconciling differences between test and analysis. This 
process requires both heuristics and quantitative methods to assess model deficiencies, to evaluate 
parameter importance, and to compute required changes. Although model update (calibration) has been 
a very prolific area of research in the US and abroad, no single technique is universally accepted and at 
times, when viewed from a deterministic viewpoint, Avitabile ’referred to it as a problem with endless 
possibilities. For years, the commercially available tools to address the update problem relied on 



sensitivity information to judge the relative importance of parameters and to assist in making model 
changes. Friswell 2 discusses many of the conventional sensitivity based approaches, some implemented 
in many commercial tools. These tools, in the hands of experienced engineers, provide heuristic 
approaches for model calibration that work very well in reconciling differences between test and analysis, 
but often provide unrealistic parameter changes and give little insight into the probabilistic nature of the 
problem. This aspect of the problem has prompted extensive research with noteworthy contributions from 
Hasselman et al 3 , Herendeen et al. 4 , Alvin 5 , Farhat and Hemez 6 , is aimed at addressing uncertainty. 
Flasselman discussed propagation of parameter uncertainty in frequency response calculations and 
presented various approaches to handle variability of response values near dynamic resonant conditions. 
Herendeen et al . 4 discussed a mathematical procedure using multi-disciplinary optimization to conduct 
analysis/test calibration studies of frequency response data. Alvin 5 extended a procedure developed by 
Farhat and Hemez 6 to improve convergence and to incorporate uncertainty information into the estimation 
process, taking full advantage of the model structure and sensitivity. To their credit, very high dimensional 
problems have been calibrated successfully using these techniques. However, in the end the question 
about realism of updated parameters is still unanswered. To properly address this question one would 
need to exploit the work that Montgomery 7 has done in terms of design of experiments and Analysis of 
Variance (ANOVA) as a means to judge parameter adequacy. In the work of Uebelhart , 8 tools from 
modern designs of experiments are relied heavily for uncertainty quantification and parameter selection. 
Regardless of the parameter selection approach, engineering judgment will always play a key role, and 
these tools are available to guide the analyst. 

In the computational fluid dynamics (CFD) area, work by Oberkampf and Roach is leading the 
verification and validation of mathematical models effort. In this area, techniques for systematic 
assessments of model adequacy and uncertainty quantification have been established along with 
standards, and a common language. Unfortunately, these efforts and the methods associated with them 
are just now slowly migrating to other disciplines; see, for example, Thacker The principles for model 
verification and validation set forth by the CFD community are applicable to many other disciplines but the 
metrics for assessment need to be modified. Using current terminology, the work presented here is 
primarily a model calibration effort. 

Fundamental to the success of the model calibration effort is a clear understanding of the ability of a 
particular model to predict the observe behavior even when the observed behavior is uncertain. The 
approach proposed in this paper is focused primarily on model calibration using parameter uncertainty 
propagation and quantification, as opposed to a search for a reconciling solution. The process set forth 
follows a two-step approach. First, ANOVA is used for parameter sensitivity, which is followed by 
uncertainty propagation to evaluate uncertainty bounds and to gage the ability of the model to explain the 
observed behavior. Once this process is completed, optimization is used to find a reconciling solution. 

This approach was demonstrated by Horta 12 . 

Key to uncertainty propagation is the calculation of both single and multi-parameter variances. 
Because of the computational burden associated with variance-based sensitivity estimates, the approach 
in this report uses response surface models to estimate the frequency response functions (FRF) from 
which the variance is computed. Response surface modeling is based on the Extended Radial Basis 
Functions (ERBF) as described by Mullur' 8 '' 4 . 

To quantify the agreement between test and analysis for dynamic problems, it is common to use 
FRF data. Normally hundreds of sensors are placed on a structure to identify frequencies and mode 
shapes. If FRFs are used as calibration metrics, the number of metrics equals the product of the number 
of sensors times the number of inputs, which is often a large number. To reduce the number of metrics 
used for comparison, the Principal Values (PV) (also known as Principal Components (PC)) of the FRF 
are used instead. Incidentally, to expand this metric across multiple models, PV uncertainty bounds for 
the maximum of all maxima and the minimum of all minima are easily computed along with the probability 
of observing such values as a function of frequency, see Horta' 5 . These bounds are then used to 
determine the probability that a solution exists that reconciles test results with analysis. To obtain a 
parameter set that reconciles the model with test, nonlinear optimization as described by Lewis ' 6 is used 
with a quadratic objective function to minimize the error between test and analysis. Hasselman 17 also 



used PC in a procedure he calls a PC-based statistical energy analysis, for generic uncertainty 
quantification and comparison of frequency responses. Our approach is very similar with the addition of 
PV maximum and minimum bounds. Future work needs to incorporate a probabilistic performance metric. 

2 DESCRIPTION OF ARES l-X VEHICLE 

The Ares l-X vehicle shown in figure 1 , consists of the First Stage (FS), Upper Stage Simulator 
(USS), and the Crew Module and Launch Abort System (CM/LAS). This demonstration vehicle uses 
heritage shuttle first stage booster technology with newly designed components to mimic the mass 
loading of the Ares I vehicle. Because heritage hardware is used, and due to scheduling issues, only two 
components of the vehicle are tested prior to the flight vehicle modal test. Stack 5 (SS5), shown in figure 
1, is the first major component tested. To minimize interference with the vehicle assembly schedule, SS5 
is tested on a non-flight interface adapter while awaiting integration into the flight vehicle. This non-flight 
boundary interface adapter, facilitates access and transportation, but is not part of vehicle. Nonetheless, 
a finite element model (FEM) of the adapter is also part of the model evaluation. 

Stack 1 (SSI ) is the second major component tested and is shown in Fig. 1 . It is comprised of the 
Interstage 2, Interstage 1, Frustum, Forward Skirt Extension, Forward Skirt, and 5 th Segment Simulator. 
This component is also tested while resting on a metal frame at the Vehicle Assembly Building (VAB) at 
the Kennedy Space Center. Because of the unconventional boundary conditions for both component 
tests, data analysis presented some unusual challenges. 

Finally, a flight vehicle modal test is completed with the vehicle resting on hold-down posts on the 
Mobile Launcher Platform (MLP), shown in figure 2. This configuration is the closest to the flight 
configuration, and therefore, it is the most relevant configuration for guidance and navigation model 
verification. 


3 ARES l-X FINITE ELEMENT MODEL DESCRIPTION 


Figure 2 shows the Ares l-X flight vehicle FEM. The FS, USS, and CM/LAS finite element models 
were delivered by independent product teams (IPT). The three FEMs are connected with rigid elements 
(RBE2) at each interface. Between each of the three FEMs there are 24 connection points. These models 
reference a global coordinate system and have a unique numbering system to simplify the model 
integration effort. The Ares l-X modeling approach is consistent with current modeling practices for this 
type of structure. The vehicle skin is modeled using shell elements (CQUAD4 and CTRIA3), and section 
flanges and support bracing are modeled using beam elements. Lump masses such as nose tip, 
engines, splice plates, and umbilical are modeled using concentrated masses and are attached to the 
vehicle using rigid bar elements (RBE2). Constraint elements (RBE3) are used to attach platforms, 
secondary structures, and concentrated masses. Joints on the upper stage and first stage segments as 
well as ground and non-flight hardware boundary interface for the SS5 and SSI test configurations are 
modeled with CBUSH spring elements. 

4 PROBLEM FORMULATION 


Model calibration begins with an initial assessment of the model adequacy for its intended use. For 
example, if the model is being developed to predict loads, detailed analysis of critically loaded regions 
must be emphasized. On the other hand, if the model is developed to support control design, critical 
structural modes must be measured and calibrated against analysis. The intended use determines the 
type of test and calibration process to follow. Metrics for the vehicle are established by the Guidance and 
Navigation Group in terms of allowable excursions from the nominal Ares l-X free-free model. 

Specifically, discrepancies in the frequency from test and analysis must be less than 10% for the 1 st 
bending pair and less than 20% for higher frequency modes; node locations for the 1 st three bending pairs 
must be within +/- 100 inches of the nominal, and modal displacement differences must be less than 20% 
for the first bending pair and 50% for higher frequency modes. Although these metrics are for the free- 



free configuration, they are also used to evaluate the vehicle model on the MLP. The ability to observe 
the modes of interest of any modal test depends heavily on proper pre-test analysis, adequate sensor 
count, and sensor/shaker placement. Readers are referred to Buehrle 18 for details on the pre-test 
analysis and test configuration. Finally, the calibration process followed during test is shown in figure 3 
Specific elements of the process are described next. 

4.1 PARAMETER SELECTION 


This step is perhaps the most difficult one because it requires first-hand knowledge of the 
assumptions and approximations used during model development. Also, because heritage hardware 
components are involved, those components and their parameters have not been considered for 
calibration; specifically, the first stage booster and the MLP. Collecting the necessary information for this 
step requires interviews with model developers and complete familiarization with all model elements.. 

After selecting an initial parameter set, the next step is the selection of parameter uncertainty models. 
These uncertainty models describe any a priori knowledge of the parameter variations in the form of 
distribution functions. In this problem, since there is no uncertainty information available, uniform 
distributions functions are used throughout the paper. As a consequence, all parameter updates resulting 
from this process are equally acceptable. Furthermore, parameter bounds are selected to assure that 
updates are acceptable from an engineering viewpoint. 

The uncertainty propagation and quantification step requires executing the FEM multiple times with 
parameter variations as prescribed by the uncertainty model. This phase reveals outcomes that are highly 
probable, determines bounds for the response of interest, and is aimed at answering the question: what is 
the probability that the model can predict the measured data? Probability assessments are all based on 
discrete probabilities computed using a prescribed number of FEM runs. For example, using solutions 
obtained from n different parameter sets, the discrete probability of observing a particular output is simply 
1/n. Our goal in selecting a parameter set is to make the probability of capturing the measured response 
high. In other words, to find a parameter set that makes the analysis bounds as encompassing as 
possible. 

Results from the parameter uncertainty study are used directly to create response surface models for 
use with a variance-based approach to determine parameter importance. Once the important parameters 
are identified, our ability to calibrate the model is assessed in terms of probability bounds. If the 
probability of finding a reconciling solution is greater than zero (i.e., test is within the uncertainty bounds), 
parameter updates are sought using the nonlinear optimization. Otherwise the parameter selection and 
bounds must be re-visited and modified. 

4.2 ANALYSIS OF VARIANCE 


Parameter sensitivity in most engineering fields is often associated with derivative calculations at 
specific parameter values. However, for analysis of systems with uncertainties, sensitivity studies are 
often conducted using ANOVA. In classical ANOVA studies, data is collected from multiple experiments 
while varying all parameters (factors) and also while varying one parameter at a time. These results are 
then used to quantify the output response variance due to variations of a particular parameter as 
compared to the total output variance when varying all the parameters simultaneously. The ratio of these 
two variance contributions is a direct measure of the parameter importance. 

19 20 22 

Sobol and others ' have studied the problem of global sensitivity analysis using variance 
measures. Sobol developed the "so called" Sobol indices to provide a measure of parameter sensitivity. 
In his formulation the variance is computed using the following expressions; 
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where fJ. is the mean value of the response of interest y, D is the total variance computed using A/ 
samples, and D x is the single parameter variance due to parameter x. Note that, v t refers to the / <h 
sample of the parameter vector and x, is a parameter within v ; . about which the variance is being 
evaluated, whereas z, are all other parameters that comprise v j not including x,. To properly evaluate Eq. 
(1), one needs at least 2 N function evaluations; one where all variables are randomly sampled, and a 
second set where all but x, are re-sampled. With this information the 1 st Sobol index is computed as 
S X =D X I D . Of course, for problems with m parameters, there are m possible first order factors. 
Equation (1) is easily extended to study two or more parameter (factor) interactions, as described in 19, 
simply by adjusting the number of parameter that gets re-sampled. Only first factors are studied in this 
paper. 

Depending on the type of structure being analyzed, FEM and their solutions can be computationally 
intensive. Because the parameter selection process relies on statistical analysis of the response data, it 
is important that no FEM solution is wasted. A way to capture information from every computed solution, 
as parameters are changed, is to use response surface models, i.e., surrogates. For this purpose, the 
Extended Radial Basis Function ERBF as developed by Mullur 13-14 is used. With this surrogate, it is now 
possible to conduct ANOVA in cases where computing thousands of FEM solutions directly is 
prohibitively expensive. 

A final note on ANOVA using Sobol’s approach regards convergence of the variance estimates. 
Although Ref. 21 discusses the asymptotic behavior of the variance estimates when using Eq. (1 ), for 
cases where a surrogate model is used instead of the FEM, the variance estimates are only as good as 
our surrogate model. Nonetheless, this approach provides an excellent way to rank variable importance 
when only a limited number of FEM solutions are possible. 

4.3 GOODNESS METRICS 


Metrics for dynamic problems are more appropriately provided in terms of mode shapes, frequencies, 
and FRF. One commonly used metric is the orthogonality criterion. This metric compares mode shapes 
extracted from measured FRF with FEM mode shapes weighted using the analytical mass matrix. 
Because this metric has been historically used, it is one of two metrics computed and reported. A second 
metric compares differences in the FRF from test and analysis directly. For optimization and uncertainty 
quantification studies this second metric is preferred and used for model calibration. 

When studying the effect of parameter changes, Sobol is used to evaluate variances in the FRF at 
each frequency due to parameter variations. Instead of comparing each input/output pair, data is 
compared in terms of the Principal Values (PV) of the FRF. These PVs are computed as the singular 
value decomposition of the FRF matrix at each frequency. For cases with multiple singular values, only 
the maximum and minimum are compared. 

5 DISCUSSION OF RESULTS 


Results are presented for the component test first to make it easier for the reader to follow. Results for 
Stack 5 are presented first followed by Stack 1 , and then the flight vehicle. 



5.1 STACK 5 MODEL CALIBRATION 


The first major component tested is the Command Module and Launch Abort System (SS5) as shown in 
figure 4. To facilitate access to the interior of the structure while on the ground, a service access structure 
(bottom yellow section) is mated and tested with the SS5 hardware, as shown in the photograph. Although 
this section is not part of the flight hardware, it is modeled, tested, and reported with the results. Figure 5 
shows pre-test prediction of the 1 st three pairs of bending modes with their corresponding frequencies. 

Results from an independent traceability study by Tuttle” 3 that considered subsystem coupling of SS5 with the 
flight vehicle concluded that although several modes of SS5 contribute to the total strain energy, the 1 st 
bending pair is the most important set and therefore should be the focus for this component test. 

Analysis of Variance for SS5- is one of two tools used for selection of critical parameters. To begin this 
analysis, a second order ERBF response surface model with 8 parameters, as defined in Table 1, is initially 
created from 400 FEM eigenvalue solutions to provide modal frequencies and mode shapes as a function of 
boundary stiffness. From these 400 runs, the FRF is synthesized for displacements outputs and is used to 
evaluate FRF variations as a result of parameter variations. Initially, FEM runs are executed with parameter 
values uniformly distributed between the bounds shown in Table 1. By construction, the ERBF surrogate 
model matches each of the 400 solutions exactly. To study parameter contributions to the FRF variance, the 
ERBF surrogate is used instead of the FEM to generate hundreds (often thousands) of predictions and to 
compute the Sobol indices according to Eq. (1). Figure 6 shows the results from the Sobol calculation as a 
function of frequency with N set to 1000 in equation 1 . Colors are used to depict contribution of the individual 
parameter to the total variance where the total contribution from all 8 parameters is less than or equal to 1 . 
Frequency values with no colors are simply areas where the variance is not computed. As an example on 
how to use this information, suppose that changes to regions lower than 14 Hz are needed, then changes to 
parameters KV1 and KV3 (vertical stiffness of quadrant I and III) will produce the largest variation in the 
maximum principal value of the FRF. Note that quadrants, labeled in figure 6 using roman numerals l-IV, 
correspond to parameters labeled 1-4. For reference, at the top of figure 6 is a plot of the maximum PV as a 
function of frequency (with the nominal model) to indicate resonant frequencies near 4 Hz, 12 Hz, and 26 Hz. 
This sensitivity information is critical to understand how to properly set parameters before any optimization is 
performed. More importantly, it provides users with a per frequency map of parameter importance. Although 
individual parameters can dominate the variance in certain frequency ranges, variance alone does not provide 
information on the magnitude of the changes. To study magnitude variations, the approach using PV bounds 
is presented next. 

SS5 Principal Value Uncertainty Analysis - The objective of this step is to assess variations in the FRF and 
to determine FRF bounds due to parameter variations. Using 400 parameters sets the maximum and 
minimum principal value across all models are computed and plotted as a function of frequency. These 
results are used not only as bounds for the measured FRF but also to estimate probabilities. Intuitively, if the 
response from 400 FEM models falls within the bounds then the probability of measuring a value outside the 
bounds must be less than 1/400, if the parameter uncertainty model is adequate. Figure 7a shows in solid- 
red the analysis bounds due to variations in the boundary stiffness computed prior to test, and the blue- 
dashed line shows the test data PV. Although the test data is from 58 accelerometers (with two shakers), the 
acceleration FRFs are scaled using frequency to compare results in terms of displacements. Incidentally, all 
analytical modes are assumed to have damping levels of 0.5%. Two observations are in order regarding the 
results shown in figure 7a: 1 ) variations in the PV values due to boundary stiffness changes are so small that 
the bounds looks just like the PV (see figure 8a), and 2) since our test data is outside the uncertainty bounds, 
the model representation is not adequate. The problem with this model is that the structure is on the ground 
without physical constraints, which makes the boundary stiffness unknown (epistemic uncertainty). It is clear 
that our initial estimate is incorrect and requires a second look. 

Because of project time constraints, model calibration for SS5 concentrated exclusively on the boundary 
stiffness as the main source of discrepancies. After a post-test look at the model, it is determined that the 
boundary stiffness is two orders of magnitude lower than initially estimated. In addition, a discrepancy was 
found involving inertia properties used to simulate the engine mass. After adjusting the boundary stiffness 
uncertainty model as shown in Table 2 and correcting for the simulated engine inertias, results with the 



updated bounds are shown in figure 7b along with the test data PV. With these corrections, the measured 
response is now within the analysis uncertainty bounds (for the 1 st two pair of modes) and parameter 
optimization can now proceed to determine stiffness parameters to reconcile test with analysis. Figure 8a 
shows the PV for the pre-test nominal model (solid-red) and test (dashed-blue); whereas figure 8b shows 
results with the optimized boundary stiffness. Although results now are more in line with the measured FRF 
data, improvements are also seen in the orthogonality results. 

Figure 9a shows orthogonality results comparing the pre-test FEM mode shapes (ordinate) with mode 
shapes identified from test (abscissa); whereas, figure 9b shows results with the optimized boundary 
parameters. When examining these results, recall that orthogonality values range from 0 to 1 with black 
squares corresponding to values of 1 ; i.e. black squares indicate exact match. It is clear that updates to the 
boundary stiffness helped correct for errors in the 1 st pair of modes principal direction as well as reducing their 
frequency error. With this correction, the 1 st mode frequencies are within 2.9%. Since the traceability study in 
Ref. 23 indicated that only the first bending mode pair is critical, no further updates are pursued. 

Because models with uncertainty are being compared to test, test uncertainty is also critical. 

Unfortunately, in order to properly address this question one would need additional data, which for economic 
reasons have not been collected. Of the small number of data sets collected, PV values for test data is 
studied to determine if there is significant variability across different tests. With the limited number of tests, 
this variability, although not shown in the paper, is not significant enough to warrant additional tests. 

5.2 STACK 1 MODEL CALIBRATION 


Stackl is the second major component tested and is shown in figure 10. In this configuration, pre-test 
predictions of target modes are shown in figure 1 1 with the boundary modeled using springs to ground. As 
with SS5, results from an independent traceability study identified modes 1, 2, 6, 7 and 8 as our target modes 
for this component test. 

SSI Analysis of Variance - begins again by creating a second order ERBF response surface model with 8 
parameters from 300 FEM eigenvalue solutions computed while varying the boundary stiffness. In this case 
the boundary is also divided into quadrants and stiffness parameters for each quadrant are varied 
independently. As before, the FRF response is synthesized for displacements outputs. To evaluate variations 
in the FRF as a result of parameter variations, FEM runs are executed with parameter values uniformly 
distributed between the bounds shown in Table 3. Variance results for Stack 1 are shown in figure 12 for 
each of the 8 boundary stiffness parameters. Again, only results for the variance of the maximum PV are 
shown; the nominal model PV is shown at the top of figure 12. In contrast to results for Stack 5, for 
frequencies less than 5 Hz, the lateral stiffness in all quadrants contribute significantly to the total variation. 

SSI Principal Value Uncertainty Analysis - For 300 parameter sets, the maximum and minimum PV 
across all FEM models are computed and plotted as a function of frequency. Figure 13a shows in solid-red 
the FEM PV bounds due to variations in the boundary stiffness computed prior to test and the blue-dashed 
line shows the test data PV. For this case, test data from 64 accelerometers (with 2 shakers) are used to 
recover the FRF and then converted to displacement FRF for comparison. It should be obvious that the 
measurements are outside the pre-test model uncertainty bounds, and consequently the model is not 
adequate. As with SS5, the boundary stiffness is found to be two orders of magnitude softer than the initial 
estimate. After adjusting the uncertainty model for the boundary, the updated parameter ranges are defined in 
Table 4. With this updated set, the computed uncertainty bounds are now shown in figure 13b; solid-red is 
analysis and dashed blue is test. Damping levels for the model are again set to 0.5 % for all modes. Since 
the boundary stiffness is unknown, it is selected in such a way as to ensure that the test data is captured 
within the bounds for the target modes. Figure 13b shows this assumption to be true; and, therefore, one can 
proceed to compute updated parameters using optimization. 


For comparison, figure 14a shows the pre-test maximum and minimum PV with the nominal model in 
solid-red versus test in dashed-blue. Similarly, Figure 14b shows the results for SSI with the optimized 



boundary parameters, shown in Table 4. Note that PV matching of test with analysis is improved significantly 
for the 1 st mode pair. 


Finally, orthogonality results for SSI are shown in figure 15a using the pre-test model and 15b uses the 
optimized boundary stiffness. Concentrating on the 1 st pair, the optimized values corrected a problem with 
the principal directions and also reduced the frequency errors. Although high frequency modes are difficult to 
observe with the limited instrumentation suite, qualitative assessments of test modes with orthogonality 
values greater than 0.8 revealed that many of these modes resemble modes found in the analysis. No further 
updates are performed due to time constraints and the need to prepare for the flight vehicle test. 

5.3 FLIGHT TEST VEHICLE (FTV) MODEL CALIBRATION 


The flight vehicle calibration process follows the same steps as those described with the component 
tests. Since the Ares l-X vehicle is tested while mounted on four hold-down posts on the MLP, as opposed to 
free-free, target modes in this configuration are the 1 st four bending mode pairs. Figure 16 shows only the first 
mode of each pair. It is assumed that after calibrating the analytical model of the vehicle on the MLP, the 
MLP can be removed analytically to predict the free-free configuration. In addition, damping values obtained 
from this test can be allocated to the free-free modes. 

FTV Analysis of Variance- Unlike the SSI and SS5 component tests, the FTV boundary condition is 
relatively well defined. Consequently, parameters selected for uncertainty analysis and their variations must 
be supported with data collected from various sources. After a careful study of potential sources of 
uncertainty, the parameters defined in Table 5 are selected. The seven parameters used in this study 
included the USS joint stiffness, FTV/MLP interface stiffness, forward skirt Young modulus XL_SK_E, the 
CM/LAS Young modulus LAS_E, as well as ballast mass densities B_US_1/B_US_7. Ranges used are 
based on collected information regarding modeling assumptions and uncertainty in the information collected. 

Figure 17 shows results for the variance analysis with the seven parameters. Note that the variance for 
the first two modes (about 0.2 Hz) is dominated by ballast densities, whereas modes between 3 to 4 Hz are 
dominated by the USS joint stiffness. 

FTV Principal Value Uncertainty Analysis - Due to time constraints, only 200 FEM runs are used to 
compute the PV bounds. Figure 18 shows results for the FEM analysis uncertainty bounds in solid-red and 
test in dashed blue. Two aspects of these results should be highlighted; 1) variations in the PV due to 
parameter variations are relatively small, and 2) the uncertainty bounds captures relatively well the measured 
data with only a few exceptions. Damping for the analysis is again assumed to be 0.5 % for all flexible 
modes. Figure 19 shows the PV using the pre-test nominal model (solid-red) as compared to the measured 
values (dashed-blue). Again the agreement is good. 

A final check examines orthogonality results of the pre-test nominal model with the identified modes from 
test. Results for this metric are shown in figure 20. Along the abscissa are test mode frequencies and along 
the ordinate are analysis frequencies. Modes circled correspond to target modes that need to be verified for 
controls. Frequency errors for those modes are under 6.5 %. Table 6 shows a summary of the identified 
frequencies and measured damping ranges (in brackets) using only random excitation data. 

With this information and the requirements initially provided by the guidance and navigation team, the 
nominal Ares l-X model is found to be adequate for evaluation of the flight control system without need for 
additional parameters adjustments. Although there are certain areas where the model can be improved, 
these areas should not impact guidance performance of the vehicle based on the initial requirements. 

6 CONCLUDING REMARKS 


A procedure to conduct model calibration for the Ares l-X vehicle has been presented The approach 
uses a variance-based approach for parameter selection, nonlinear optimization to minimize the error 
between test and analysis, and multiple FEM models to bound the system response and to assess the 


probability of finding a reconciling solution. To alleviate the computational expense in performing variance- 
based sensitivity analysis, the approach uses a surrogate model to predict changes in the frequency response 
functions as parameters are varied. Uncertainty in the parameters and their effect on the frequency response 
function is studied in terms of Principal Values. Uncertainty bounds of the principal values are established 
across multiple models to allow one to determine the probability of finding a solution that reconciles test with 
analysis results. Model adequacy is easily ascertained by comparing the measured responses against PV 
uncertainty bounds. 

Because of the use of unconventional boundary conditions during Stack 5 and Stackl tests, the model 
calibration effort concentrated on boundary stiffness uncertainty characterization. In both cases, the initial 
boundary stiffness estimates were two orders of magnitude lower than initially thought. After adjusting the 
uncertainty models for the boundary stiffness to properly capture the measured data, optimization is used to 
find a set of parameters that reconciles test with analysis. This optimized parameter set is found to minimize 
the error between the test and analysis FRF. Because uncertainty data is not available, it is impossible to 
make probabilistic statements regarding confidence in the optimized boundary stiffness values. Although, the 
boundary stiffness parameters do not alter vehicle parameters directly, agreement of test with analysis for the 
target modes improved significantly after boundary adjustments. Likewise, our confidence in the nominal 
model is also improved. 

Finally, results for the Ares l-X vehicle are presented and compared to predictions using the nominal 
model with uncertainty bounds computed prior to test. These results show that the nominal model is 
remarkably close to the pre-test predictions and complies with model verification metrics provided by the 
guidance and navigation group. For this reason, changes to the nominal model are not recommended. 

7 ACKNOWLEDGEMENTS 

Special thanks to the FEM development group for their continued support: Winifred Feldhaus 
(LaRC.FTV), Genevieve Dixon (LaRC, CM/LAS), Troy Mann (LaRC, CM/LAS), Richard Pappa 
(LaRC,CM\LAS), Nickolas A. Vitullo (ATK, weight), Andrew Panetta (LaRC,weight/cg), William K. Thompson 
(GRC, upper stage) , Jeremy Redden (ATK, first stage) , Carl Pray (ATA, uncertainty analysis, Jones Trevor 
(GRC, upper stage) , Jennifer Lange (GRC, upper stage). Thanks to our independent verification team of Jeff 
Lollock, Ryan Tuttle, and Joshua Flwung from Aerospace Corporation. Finally thanks to the KSC team; Russ 
Brucker, Trip Healey, Frank Walker, Stephanie Heffernan, Teresa Kinney, Todd Reeves, Kara Schmitt, Mark 
Tillett, and Karl Meyer. 


8 REFERENCES 


1. Avitabile, P.\ “Model Updating- Endless Possibilities.” Proceedings of the 18 th International Modal Analysis 
Conference, San Antonio, TX, Feb. 2000, pp. 562-570. 

2. Friswell, M.I., and Mottershead, J.E.: Finite Element Model Updating in Structural Dynamics, Kluwer 
Academic Press, Dordrecht, 1995. 

3. Hasselman, T.K., Chrostowski, J.D., and Ross, T.J.: “Propagation of Modeling Uncertainty Through Structural 
Dynamic Models,” Proceedings of the 35 th AIAA/ ASME/ASCE/AHS/ASC Structures, Structural Dynamics, 
and Materials Conference, 1994, Paper No. AIAA 94-1316, SC. 

4. Herendeen, D.L., Woo, L., Hasselman, T.K., and Zimmerman, D.C.: “Analysis-Test Correlation and Model 
Updating of Dynamic Systems Using MDO Software Tools,” Proceedings of the 7 th AIAA/USAF/NASA/SSMO 
Symposium on Multidisciplinary Analysis and Optimization, 1998, Paper No. 98-4730, St. Louis, MO. 

5. Alvin, K.F.: “Finite Element Model Update via Bayesian Estimation and Minimization of Weighted Residuals,” 
AIAA Journal, Vol. 35, No. 5, 1997. 

6. Farhat, C. and Hemez, F.M.: “Updating Finite Element Dynamic Models Using an Element-by-Element 
Sensitivity Methodology,” AIAA Journal, Vol. 31, No. 9, 1993, pp. 1702-1711. 

7. Montgomery D.C., Design and Analysis of Experiments . John Wiley & Sons, New York, 5 th ed. 2001 

8. Uebelhart, S.A., Miller, D.W., and Blaurock, C.: “Uncertainty Characterization in Integrated Modeling.” 
Proceedings of the 46 th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, & Materials 
Conference, April 2005, Austin TX., AIAA 2005-2142. 



9. Oberkampf, W. L., Trucano, T.G., and Hirsch, C.: “Verification, Validation, and Predictive Capability in 
Computational Engineering and Physics.” SAND 2003-3769, Feb. 2003. 

10. Roach, P.P., Verification and Validation in Computational Science and Engineering . Hermosa Publishers, 
Albuquerque, NM, 1998. 

11. Thacker, 6.H.: “The Role of Non-determinism in Computational Model Verification and Validation.” 
Proceedings of the 46 th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, & Materials 
Conference, April 2005, Austin TX, AIAA 2005-1902. 

12. Horta, L.G., Reaves, M.C., and Lew, J.-S.: “A Procedure for Static and Dynamic Model Update of Finite 
Element Models: Application to an Inflated/Rigidized Torus.” Proceedings of the IMAC XXIV: Conference and 
Exposition on Structural Dynamics, 2006, St. Louis Missouri. 

13. Mullur, A. and Messac, A.: “Extended Radial Basis Functions: More Flexible and Effective Metamodeling.” 
AIAA Journal, Vol., 43, No. 6, June 2005. 

14. Mullur, A. and Messac, A.: “Metamodeling Using Extended Radial Basis Functions: A Comparative 
Approach.” Engineering with Computers (2006) 21: 203-217 DOI 10.1007/s00366-0005-7. 

15. Horta, L.G., Kenny, S.P., Crespo, L.G., and Elliott, K.B.: “NASA Langley’s Approach to the Sandia’s Structural 
Dynamics Challenge Problem.” Computer Methods in Applied Mechanics and Engineering, Vol. 197, Issues 
29-32, May 2008, also available at www.sciencedirect.com. 

16. Lewis. R.W., Shepherd A., and Torczon, V.: “Implementing Generating Set Search Methods for Linearly 
Constrained Minimization.” SIAM Journal on Scientific Computing, Vol. 29, No. 6, pp 2507-2530. 

17 . Hasselman T, Yap K., Yan H. and Parret, A.: “Statistical Energy Analysis by Principal Components for Mid- 
Frequency Analysis.” Proceedings of the 43 rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, 
and Materials Conference, April 2002, Denver, CO. AIAA 2002-1395. 

18. Buehrle, R.D., Templeton, J.D., Reaves, M.C., Horta, L.G., Gaspar, J.L, Bartolotta, P.A., Parks, R.A., and 
Lazor, D.R,: “Ares l-X Launch Vehicle Modal Test Overview.” Proceedings of IMAC XXVIII, Jacksonville, FL., 
February 1-4, 2010. 

19. Sobol, I.M., Tarantola, S., Gatelli, D., Kucherenko, S.S, and Mauntz, W.: "Estimating Approximation Error 
When Fixing Unessential Factors in Global Sensitivity Analysis," Reliability Engineering and System Safety 92 
(2007) 957-960, © 2006 Elsevier LTD. 

20. Mullershon, H., and Liebsher, M.: “Statistics and Non-Linear Sensitivity Analysis with LS-OPT and DSPEX.” 
10 th International LS-DYNA Users Conference, pp. 4-1,4-13. 

21 . Homma, T, and Saltelli, A.: “Importance Measures in Global Sensitivity Analysis of Nonlinear Models.” 
Reliability Engineering and System Safety 52 (1996) 1-17, © 1996 Elsevier LTD. 

22. Sudret, B.: "Global Sensitivity Analysis Using Polynomial Chaos Expansion," Reliability Engineering and 
System Safety 93 (2008) 964-979, © 2007 Elsevier LTD. 

23. Tuttle, R., Lollock, J.A, and Hwung, J.S.: “Identifying Goals for Ares l-X Modal Testing,” Proceedings of IMAC 
XXVIII, Jacksonville, FL., February 1-4, 2010. 



No. 

Parameter ID 

Lower 

Bound 

Nominal 

Upper 

Bound 

1 

Kvl (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

2 

KI1 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

3 

Kv2 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

4 

KI2 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

5 

Kv3 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

6 

KI3 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

7 

Kv4 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 

8 

KI4 (lbs//in) 

4.84E+07 

6.06E+07 

6.06E+07 


Table 2 Stack 5 Post-test parameter definition and optimized solution 


No. 

Parameter ID 

Lower Bound 

Nominal 

Upper Bound 

Optimized 

Solution 

1 

Kvl 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

5.45E+05 

2 

KI1 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

6.01 E+05 

3 

Kv2 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

3.33E+06 

4 

KI2 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

1.09E+06 

5 

Kv3 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

5.47E+05 

6 

KI3 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

5.94E+05 

7 

Kv4 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

3.33E+06 

8 

KI4 

(lbs//in) 

5.45E+05 

6.06E+05 

3.33E+06 

3.33E+06 


Table 3 Stack 1 pre-test parameter definition 


No. 

Parameter ID 

Lower Bound 

Nominal 

Upper Bound 

1 

KQ2RTh (Ibs/in) 

1.00E+06 

1.00E+07 

1.00E+08 

2 

KVQ2 (Ibs/in) 

1.00E+06 

1.00E+07 

1.00E+08 

3 

KQ3RTh (Ibs/in) 

1.00E+06 

1 .00E+07 

1.00E+08 

4 

KVQ3 (Ibs/in) 

1.00E+06 

1.00E+07 

1.00E+08 

5 

KQ4RTh (Ibs/in) 

1.00E+06 

1 .00E+07 

1.00E+08 

6 

KVQ4 (Ibs/in) 

1.00E+06 

1.00E+07 

1.00E+08 

7 

KQIRTh (Ibs/in) 

1.00E+06 

1 .00E+07 

1.00E+08 

8 

KVQ1 (Ibs/in) 

1.00E+06 

1.00E+07 

1.00E+08 


Table 4 Stack 1 Post-test parameter definition and optimized solution 


No. 

Parameter 

ID 

Lower 

Bound 

Nominal 

Upper 

Bound 

Optimized 

Solution 

1 

KQ2RTh (Ibs/in) 

2.50E+05 

3.75E+05 

1.00E+06 

1.00E+06 

2 

KVQ2 (Ibs/in) 

2.50E+05 

3.75E+05 

1.00E+06 

8.03E+05 

3 

KQ3RTh (Ibs/in) 

2.50E+05 

3.75E+05 

1 .00E+06 

1.00E+06 

4 

KVQ3 (Ibs/in) 

2.50E+05 

3.75E+05 

1.00E+06 

2.50E+05 

5 

KQ4RTh (Ibs/in) 

2.50E+05 

3.75E+05 

1 .00E+06 

1.00E+06 

6 

KVQ4 (Ibs/in) 

2.50E+05 

3.75E+05 

1.00E+06 

6.32E+05 

7 

KQIRTh (Ibs/in) 

2.50E+05 

3.75E+05 

1 .00E+06 

1.00E+06 

8 

KVQ1 (Ibs/in) 

2.50E+05 

3.75E+05 

1.00E+06 

4.27E+05 


Table 5 Flight vehicle pre-test parameter ranges 


No. 

Parameter 

ID 

Lower 

Bound 

Nominal 

Upper 

Bound 

1 

USS_J (Ibs/in) 

4.84E+06 

1.00E+07 

2.63E+07 

2 

FTV/MLP_V (Ibs/in) 

2.43E+08 

2.70E+08 

2.97E+08 

3 

FTV/MLP_L (Ibs/in) 

2.43E+08 

2.70E+08 

2.97E+08 

4 

B_US_1 (lbs-s 2 /in 4 ) 

6.44E-04 

7.32E-04 

8.27E-04 

5 

B_US_7 (lbs-s 2 /in 4 ) 

5.63E-04 

7.32E-04 

8.20E-04 

6 

XL_Fwd_Skirt_E (Ibs/in 2 ) 2.70E+07 

3.00E+07 

3.30E+07 

7 

LAS_E (Ibs/in 2 ) 

8.91 E+06 

9.90E+06 

1.09E+07 


Table 6 Summary of flight vehicle identified frequencies and damping 


Mode 

Freq. (Hz) 

Damp. (%) 


1 

0.18 

[0.48, 0.83] 

Y-X 1st bend 

2 

0.21 

[0.36, 0.44] 

Z-X 1st bend 

3 

1.06 

[0.28, 0.35] 

Y-X 2nd bend 

4 

1.19 

[0.39, 0.43] 

Z-X 2nd bend 

5 

1.87 

[2.22, 2.95] 

Z-X bend &MLP 

6 

2.07 

[1.28, 1.43] 

Y-X bend & MLP 

7 

2.91 

[0.96, 1.40] 

3rd bend % MLP 

8 

3.46 

[0.43, 0.52] 

3rd Y-X bend 

9 

3.64 

[0.37, 0.41] 

3rd Z-X bend 

10 

3.68 

[0.38, 0.41] 

Torsion 

11 

4.61 

[0.17, 0.18] 

4th bend (45deg) 

12 

4.78 

[0.23, 0.25] 

4th bend X-Y 

13 

6.17 

[1.35, 1.38] 

system 

14 

6.41 

[0.66, 0.69] 

system 

15 

6.66 

[1.29, 1.40] 

system 


CM / LAS 
Simulator 


Upper Stage 
Simulator 


First Stage 



Stack 5 Ares l-X Partial Stack 



Test Configurations Stack 1 
Interstage 2 
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Figure 1 Ares l-X vehicle schematic and subcomponents tested 
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Figure 2 Finite element model schematic of the flight vehicle 



Figure 3 Model calibration process 


Variance Contribution 


SS5 Test Hardware 


FEM Model 




Figure 4 Ares l-X Command Module and Launch Abort System (CM-LAS) 
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Figure 5 Pre-test predictions of first three pairs of bending modes 





Freauencv 


Figure 6 Parameter contributions to variance of the 1 st principal value due to longitudinal and vertical 
boundary stiffness variations. 







Figure 7 Comparison of SS5 principal value bounds for a) pre-test model and b) after updates to 
parameter ranges 




b) optimized 


Figure 8 Comparison of SS5 principal values for a) pre-test model b) model with optimized boundary 
parameters 


Frequency (Hz) 




0.3 

0.2 

0.1 


Frequency (Hz) 




0.4 

0.3 

0.2 

0.1 
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Figure 9 SS5 orthogonality results with a) pre-test model and b) with optimized boundary parameters 



Figure 10 Ares l-X SSI test article 
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Figure 1 1 SSI pre-test predictions 



Figure 12 SSI parameter contributions to variance of the 1 st principal value due to boundary stiffness 
variations 






a) Pre-test model bounds b) updated bounds 

Figure 13 Comparison of SSI PV bounds for a) pre-test model and b) updated boundary stiffnes bounds 




b) optimized boundary 


Figure 14 Comparison of SSI PV with a) pre-test model and b) optimized boundary stiffness model 
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Figure 15 SSI orthogonality results with pre-test model and with updated boundary stiffness model 
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Figure 16 Flight vehicle target modes 
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Figure 17 Parameter contributions to variance of the 1 st PV for the flight vehicle 



Figure 18 Comparison of flight vehicle PV bounds for pre-test model and test (assumed damping for 
model is 0.5 %) 






Figure 19 Comparison of flight vehicle PV for pre-test model and test 
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Figure 20 Flight test vehicle orthogonality results with pre-test model 


